Camera for selecting an image from a plurality of images based on a face portion and contour of a subject in the image

ABSTRACT

There is disclosed a camera which detects a face portion of a subject from a shot image to determine whether a shot image is a good shot candidate image from the features of the face portion and a face portion-related image portion. As examples, the features include, but not be limited to, (1) the shape and variation in contour ranging from the vicinity of the face portion, (2) the position of a hand or leg to be determined by similarity to the skin tone of the face portion, and (3) determination made from the detection result of the face portion as to whether the subject is a half-length or full-length figure.

CROSS-REFERENCE TO RELATED APPLICATION

This application is based upon and claims the benefit of priority fromprior Japanese Patent Application No. 2006-027599, filed on Feb. 3,2006, the entire contents of which are incorporated herein by reference.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The present invention relates to an improvement of a digital camera, andmore particularly, to a camera capable of shooting with less mistakes.

2. Description of the Related Art

There is widely known a technique for detecting a human face from animage to use it for various purposes. For example, Japanese PatentApplication Laid-Open No. 2004-40491 proposes use of the size of a faceto measure the distance between a subject and equipment in order tochange the loudness level. Further, a camera with a function forfocusing and exposure control to a face portion in an image is currentlyon the market.

In addition, a technique for detecting a posture of a person is known(for example, see Japanese Patent No. 2627483).

BRIEF SUMMARY OF THE INVENTION

The camera of the present invention is to detect a face portion of asubject from a shot image and determine whether the shot image is a goodshot candidate image based on the features of the face portion and otherimage portions associated with the face portion. Examples of thefeatures of other image portions associated with the face portioninclude, but not be limited to, (1) the shape and variation in contourranging from the vicinity of the face portion, (2) the position of ahand or leg to be determined by similarity to the skin tone of the faceportion, and (3) determination made from the detection result of theface portion as to whether the subject is a half-length or full-lengthfigure.

Exemplary structural elements of the camera of the present invention canbe represented as: a face portion detection part for detecting a faceportion included in an image signal from the image signal output from animage pickup device; and a determination part for determining a shotimage as a good shot candidate image from the position of the faceportion in a frame of the image and when a contour running toward theoutside of the frame from the vicinity of the face portion is formedinto a predetermined shape.

Another exemplary structural elements of the camera of the presentinvention can be represented as: a face portion detection part fordetecting a face portion included in a shot image; a skin tone detectionpart for detecting a skin tone of the face portion detected by the faceportion detection part; and a determination part for determining a handposition in the shot image according to the detection result of the skintone detection part.

According to the present invention, a technique can be provided whichallows a user to get a good shot of a person comfortably by identifyingthe position of a face of a subject without missing the opportunity toget the best shot in the optimum scene or composition.

BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

These and other features, aspects, and advantages of the apparatus andmethods of the present invention will become better understood withregard to the following description, appended claims, and accompanyingdrawings where:

FIG. 1 is a block diagram showing the basic structure of a cameraaccording to each of preferred embodiments of the present invention;

FIGS. 2A to 2C are views showing examples of photos as good shots takenof moving subjects;

FIGS. 3A to 3C are views showing contours radially extended form thevicinity of a face and a face portion obtained as a result of contourdetection in each scene corresponding to each of FIGS. 2A to 2C;

FIGS. 4A to 4C are views showing a case where the feature of a good shotis made distinct from a previous or following related frame;

FIG. 4D is a view showing an example when shooting is done with adesired subject at the center;

FIGS. 5A to 5C are views for explaining the moment that a subject kicksa ball;

FIGS. 6A to 6C are views for explaining an example for obtaining a goodshot at the moment when the subject kicks the ball;

FIGS. 7A to 7B are views for explaining another example for obtaining agood shot at the moment when the subject kicks the ball;

FIG. 8 is a flowchart for explaining the shooting operation of thecamera according to the first embodiment of the present invention;

FIG. 9 is a flowchart for explaining the shooting operation of thecamera according to the first embodiment of the present invention;

FIG. 10 is a flowchart for explaining the shooting operation of a cameraaccording to a second embodiment of the present invention;

FIGS. 11A to 11D are views for explaining face portion detection in thesecond embodiment;

FIG. 12 is a view for explaining determination as to whether both handsof the subject are folded;

FIG. 13 is a view for explaining a subroutine “full-length figuredetermination” in step S43 of the flowchart of FIG. 10;

FIG. 14 is a flowchart for explaining the details of the subroutine“full-length figure determination” in step S43 of the flowchart of FIG.10;

FIG. 15 is a timing chart showing the timings of image capturing andgood shot determination;

FIG. 16 is a flowchart for explaining the shooting operation of a cameraaccording to a third embodiment of the present invention;

FIGS. 17A to 17C are views showing display modes for displaying an imagedetermined to be a good shot and the other images;

FIG. 18 is a flowchart showing an example of a subroutine “priorityimage selection” in step S79 of the flowchart of FIG. 16; and

FIG. 19 is a flowchart showing another example of the subroutine“priority image selection” in step S79 of the flowchart of FIG. 16.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

Preferred embodiments of the invention are described below withreference to the accompanying drawings.

First Embodiment

FIG. 1 is a block diagram showing the basic structure of a cameraaccording to a first embodiment of the present invention. This cameraincludes a main CPU (hereinafter abbreviated as “MPU”) 11, a pluralityof switches 11 a, 11 b, and 11 c, a signal processing part 12, a facedetection part 13, a contour detection part 14, a flash ROM 15, anassist light emitting part 16, a focusing part 18, a zoom control part19, an analog front-end part (AFE) 21, an SDRAM 22, a memory 23, amicrophone 24, a speaker 26, a display part 27, a print signal outputpart 28, a radio communication part 29, an image pickup device 31, and aphotographing lens 32.

In FIG. 1, the MPU 11 functioning as a control part and a determinationpart consists of a micro-controller and the like. The MPU 11 detectsvarious user's operations according to the states of the switches 11 a,11 b, and 11 c connected thereto. Further, the MPU 11 sequentiallycontrols each of the blocks to be described later to execute a shootingor playback sequence from the detection results of the switches 11 a, 11b, and 11 c according to a program stored in the flash ROM 15.

The switches 11 a to 11 c function as a power switch (11 a) forswitching the power on and off, a release switch (11 b), and switches(11 c) as a selection part for shooting, playback, and other operationsto be described later. In other words, the switches 11 a to 11 c acceptuser's operations.

Connected also to the MPU 11 are the signal processing part 12, the facedetection part 13, the contour detection part 14, the flash ROM 15, theassist light emitting part 16, the focusing part 18, and the zoomcontrol part 19.

The signal processing part 12 performs processing such as colorcorrection and signal compression. The signal processing part 12 has anauxiliary block 12 a therein. The auxiliary block 12 a is to boost thesensitivity after the exposure level is changed, change compensationvalues during image processing, and change compression rates.

The face detection part 13 analyzes an image signal obtained from theimage pickup device 31, and if a person is included in subjects, itdetects the person's face using the features of a human face. Then, theposition and size of the detected face is determined. The contourdetection part 14 has a function for determining the shape of contoursof a subject in an image obtained by enhancing the sharpness of theimage.

The flash ROM 15 stores various programs executed by the MPU 11 of thecamera, and various adjusting values related to the camera. The assistlight emitting part 16 emits assist light toward a subject 34 tocompensate for exposure or capture shadows. The focusing part 18controls the driving of the photographing lens 32 to focus the subject34 on an imaging surface of the image pickup device 31 through thephotographing lens 32 in a manner to be described later. The zoomcontrol part 19 controls the photographing lens 32 to change the angleof view.

The signal processing part 12 is connected with the analog front-endpart 21, the SDRAM 22, the memory 23, the microphone 24, the speaker 26,the display part 27, the print signal output part 28, and the radiocommunication part 29. The image pickup device 31 for forming an imageof the subject 34 from light incoming through the photographing lens 32is connected to the analog front-end part 21. This image pickup device31 consists of a CCD or CMOS sensor and the like to photoelectricallyconvert the subject image into an image signal.

The analog front-end part 21 performs reading and A/D conversion of theconverted image signal. This analog front-end part 21 also performsvarious control operations such as to change the exposure level of theinput image. The digital image signal converted by this analog front-endpart 21 is input into the signal processing part 12 in which the digitalimage signal undergoes various image processing to create image data.

This image data is recorded in the removable memory 23 as a recordingmedium. The image data to be recorded is compressed to an appropriatedata size according to the setting of the camera before being recordedin the memory 23. Upon recording, since various calculations are made,data are stored in the SDRAM 22 on an as-needed basis. In theembodiment, images (including video or moving images) are first storedin the SDRAM 22, and after completion of the calculations, the data arestored in the memory 23.

The display part 27 constituting a classification display part togetherwith the MPU 11 is composed of an LCD and the like. Signals from theimage pickup device 31 are sequentially displayed on the display part 27to allow the user to view the subject 34 in order to decide acomposition. Upon playback display, compressed image data stored in thememory 23 is read and converted to display data by the signal processingpart 12 according to a user's operation of the switch 11 c, therebydisplaying a display image on the display part 27. Similarly, forplayback of recorded audio, compressed audio data stored in the memory23 is read and converted by the signal processing part 12 according to auser's operation of the switch 11 c, thereby providing audio output fromthe speaker 26. The memory 23 can be either a built-in type or aremovable type such as a memory card.

The shot images and the like can be sent to external equipment (notshown) through the print signal output part 28 or the radiocommunication part 29.

In the embodiment, independent circuit units including ASICs areemployed for the analog front-end part 21, the face detection part 13,the contour detection part 14, the assist light emitting part 16, thefocusing part 18, and the zoom control part 19. However, the functionsof these parts can be integrated into one or more units, or part or allof the functions can be implemented as programs executed on the MPU.

In such a structure, if the user operates any of the switches 11 a to 11c, the MPU 11 detects the state of the operated switch and sequentiallycontrols each of the blocks according to a program stored in the flashROM 15 to execute a shooting or playback sequence.

Upon shooting, an image of the subject 34 incoming through thephotographing lens 32 is captured and converted to an electric signal bythe image pickup device 31, and corresponding digital image data isinput into the signal processing part 12 through the analog front-endpart 21. Then, the signal processing part 12 performs processing such ascolor correction and image compression, and the resulting data isrecorded in the memory 23, thus completing the shooting.

According to the present invention, a technique to be described later isapplied to the camera having the aforementioned structure to provide acamera suitable for capturing moments, for example, the moment that asubject 41 strikes a golf ball 43 with a golf club 42 as shown in 2A,the moment that a subject 45 reaches the goal line at an track and fieldevent as shown FIG. 2B, the moment that a subject 46 kicks a soccer ball47 at a soccer game as shown in FIG. 2C, etc.

When shooting such a subject, it is very difficult to capture thesubject at desired timing. Therefore, in the embodiment, a sequentialshooting mode for continuously shooting a plurality of images is so usedthat the user can select a desired photo from the sequentially shotimages.

However, as the number of sequentially shot images increases, not onlydoes the selection of the best shot from the shot images require moretime and effort, but also the available memory space inside the cameracould be fully occupied by the data. Therefore, it is desirable toselect only desired images automatically or semi-automatically to recordonly necessary images. To this end, the embodiment employs a method ofanalyzing images to select the desired images in consideration of a poseof the subject.

The characteristic points of the images as shown in FIGS. 2A to 2C arethat the face of a person as the subject is in a proper position, theperson's hand or tiptoe exists in a predetermined position or theperson's arms or legs stretch out, and the person has just hit a ball orthe like or the ball has just deformed on the impact.

In the embodiment, the images are narrowed down based on theseconditions to select desired scenes as candidates from lots of images.

FIGS. 3A to 3C are views in which face portions and contours radiallyextended from the face portions obtained as a result of contourdetection are extracted from the respective scenes corresponding toFIGS. 2A to 2C. In each of FIGS. 3A to 3C, a hollow arrow(s) indicatesthat a contour(s) is radially extended from each face portion markedwith a double circle, with each tip of the contour marked with a singlecircle. In case of FIG. 3A, radial contours 41 b are extended toward aball 49 located directly below a face portion 41 a. In case of FIG. 3B,radial contours 45 b 1 and 45 b 2 toward both hands 50 a 1 and 50 a 2are located above a face portion 45 a. In case of FIG. 3C, a leg (aradial contour 46 b) stretches from a face portion 46 a to the positionof a ball 51. These are examples of characteristic compositions. Inother words, if the four limbs stretch out or take up almost the wholeframe, it can be an attractive image with full of variety. The selectionof such an image is likely to meet the user's preferences, that is, theimage is more likely to be a good shot.

Further, as shown in FIGS. 4A to 4C, there is a case where a good shothas a distinctive feature that is different from previous and/orfollowing frames in a sequence of shots, rather than a feature unique toeach frame.

Suppose that a subject 53 is running in a scene of a field and trackrace as shown in FIGS. 4A to 4C. In this case, the subject 53 is runningwith his arms below the shoulder level before reaching the goal as shownin FIGS. 4A and 4B. Then, at the moment of reaching the goal in firstplace, since the subject 53 is likely to change his motion in adiscontinuous manner such as to stretch his arms high in the air asshown in FIG. 4C, this shot can be selected as a good shot. However, ifthe user wants to take a picture of a subject (person) 54 other than theperson as the first finisher, the user should keep the subject 54 at thecenter of a screen as shown in FIG. 4D. In this case, it can be set thatthe shot as shown in FIG. 4D is eliminated from selection.

Suppose further that the user wants to capture the moment that a subject56 kicks a ball 57 as shown in FIGS. 5A to 5C. In this case, the ball 57appears in frames shown in FIGS. 5A and 5B, but disappears from a frameshown in FIG. 5C. Therefore, the frame immediately before the ball 57disappears, the frame in FIG. 5B in this case, can be deemed as a goodshot.

Thus, since the good shot candidate is selected in comparison with theprevious and following frames, a more effective and strong shot can beselected.

In a practical sense, in the scene as shown in FIG. 6A, a radial contourdoes not run straight from a face portion 46 a to a tiptoe (ball 51) asshown in FIG. 6C. However, as shown in FIG. 6B, if the angle θ between acontour l₁ radially extended from the face portion 46 a and thefollowing contour l₂ is predetermined degrees or more to make thecontours look like a straight line, it can be set that the contours aredeemed to be a string of radial contours. Thus, a powerful scene of thesubject 46 stretching his leg out can be selected.

On the other hand, as shown in FIG. 7A, a scene of a subject 58 liftinghis leg high also looks powerful. In this case, from a point of viewsimilar to that shown in FIG. 6C, if the contour 12 is detected inadvance in previous scenes to track the tip of the contour across theprevious scenes, a scene in which the tip of the contour comes to aposition above the head can be selected as a good shot candidate.

Next, based on the above-mentioned point of view, the shooting operationof the camera in the embodiment will be described with reference toflowcharts of FIGS. 8 and 9.

FIG. 8 shows processing when a shooting mode is selected with anoperation of the switch 11C. When this sequence is started, the state ofthe release switch 11 b is first determined in step S1. In other words,it is determined whether the user or photographer has pressed therelease switch 11 b. If the release switch 11 b is pressed, the MPU 11detects the press, and the camera enters the shooting procedure startingfrom step S2. On the other hand, if the release switch 11 b is notpressed, the procedure shifts to step S27 to be described below.

In FIG. 9, it is determined in step S27 whether the operation mode is aplayback mode from the state of the switch 11 c. If it is not theplayback mode, the procedure goes to step S28 in which signals outputfrom the image pickup device 31 are displayed in real time on thedisplay part 27. The user decides the timing of pressing down therelease switch 11 b while viewing the images displayed on the displaypart 27.

During display of real-time images, detection of a face portion isperformed in steps S29 to S31 before shooting. This detection of theface portion is performed using a known technique for analyzing anddetecting the arrangement of subject's eyes, ridge of nose, and mouth,such as pattern matching. Alternatively, a pattern of light emitted fromthe assist light emitting part 16 and reflected from the subject can bedetected by a sensor such as the image pickup device 31. Further, thesemethods can be used together.

Thus, since the face portion is detected before actual shooting,processing after the release switch is pressed can be speeded up.

When the face portion is detected in step S29, a face portion frame (notshown) appears in a corresponding position on the display part 27 toinform the user that the face portion is detected correctly. Further, instep S31, the coordinate position (the position of the face beingdisplayed) is monitored. This allows the user to identify the positionof the face portion on the spot upon shooting.

Under this condition, when the user operates the camera to startshooting, the procedure shifts to step S2 in FIG. 8 to determine whetherthe face has already been detected. If the face has not been detectedyet, the procedure goes to step S3 in which average shooting isperformed to make the entire area of the image proper on average withoutputting emphasis on the face portion. After that, the procedure shiftsto step S23.

On the other hand, if it is determined in step S2 that the face portionhas already been detected, the procedure goes to step S4 in whichoptimum shooting for the face portion is performed to adjust theexposure and focus on the face portion. Then, in steps S5 and S6, theface coordinates are determined and the resulting coordinates arerecorded for each frame. Then, in step 7, it is determined whether thesize and position of the face is appropriate or not. If it is notappropriate, since it means that the shot can not be considered a goodshot, the procedure shifts to step S23 to be described later. On theother hand, if the size and position of the face is appropriate, theprocedure goes to step S8 in which the signal processing part 12performs sharpness processing or the like as image processing to enhancethe edges of the image.

Then, in step S9, a contour(s) radially extended from the vicinity ofthe face recorded in step S6 is detected and selected. Then, in stepsS10 and S11, the coordinates of the tip of the contour(s) (for example,see ball 49 in FIG. 3A, hands 50 a ₁ and 50 a ₂ in FIG. 3B, or ball 51in FIG. 3C) are determined and recorded. Through such a process, thecoordinates of subject's face and limb tip (see FIG. 8) are recorded indue order. If changes in these coordinate positions are tracked todetermine whether each image is good shot or not, such a scene as shownin FIG. 7A can be determined as a good shot.

Then, in step S12, a change in the contour determined in step S11 as aradial contour is determined. For example, if the change in the contouris larger than a reference value like the change from FIG. 4B to FIG.4C, the procedure goes to step S14, while it is smaller, the proceduregoes to step S13 to set a change flag “0”. Here, the positioncoordinates of the radial contour tip in step S10 and/or the amount ofchange in the radial contour in step S11 can be prerecorded in the flashROM 15 to determine the magnitude of the change in the contour based onthe prerecorded data.

If the change in the contour is large, the change flag is set to “1” instep S16 in a manner to be described later. This shooting sequence isrepeated to perform sequential shooting until the user operates thecamera to instruct a terminate operation to turn the release off. Duringthis repetition, if the change flag is changed from “0” to “1”, thepresence or absence of this change is then determined in step S14. Ifthe change flag is changed, the image for which the change flag ischanged is selected in step S15 as a candidate image. On the other hand,if there is no change in the change flag, step 15 is skipped. Afterthat, the change flag is set to “1” in step S16. Thus, in theembodiment, when the contour radially extended from the face portion ischanged by an amount more than the reference value, that is, when thechange flag is changed, the image for which the change flag is changedis selected as a candidate image.

Further, in operation steps starting from step S17, it is alsodetermined whether the subject's pose is appropriate or not. In stepS17, it is determined whether the coordinate position of the radialcontour tip (the position of a limb tip) lies above the face recorded instep S6 like the scenes shown in FIGS. 2B and 7A. As a result, if thecoordinate position of the radial contour tip lies above the face, theprocedure goes to step S18 to select the image as a good shot candidateimage. If the coordinate position of the radial contour tip does not lieabove the face, step S18 is skipped.

Then, in step S19, it is determined whether the coordinate position ofthe radial contour tip lies directly below the face coordinates like thescene shown in FIG. 2A. As a result, if the coordinate position of theradial contour tip lies directly below the face coordinates, theprocedure goes to step S20 to select the image as a good shot candidateimage. If the coordinate position of the radial contour tip does not liedirectly below the face coordinates, step S20 is skipped.

Then, in step S21, it is determined whether the shape of an objectadjacent to the radial contour tip (for example, the shape of the ball57 in FIGS. 5A and 5B) is changed or not. As a result, if the shape ofthe object adjacent to the radial contour tip is changed, the proceduregoes to step S22 to select the image as a good shot candidate image. Ifthe shape of the object adjacent to the radial contour tip is notchanged, step S22 is skipped.

Thus, through the operation steps S17 to S22, powerful scenes of thesubjects stretching his arms or leg high in the air, tense scenes ofsubjects staying focused on a play or the like with their armsstretching to a point directly below their faces, or images of themoments when subjects have just struck or hit a ball or the like can berecorded as good shot candidate images. The determination in each ofsteps S17, S19, and S21 is made, for example, by prestoring templatedata having information on each coordinate position or each area in theflash ROM 15 to compare each image with the prestored template data. Anyother techniques capable of making the determination in a like mannercan also be employed.

Then, after completion of the determination of the subject's pose, it isdetermined in step S23 whether the shooting terminate operation isinstructed. If it is not instructed, the procedure returns to step S4 torepeat the processing starting from step S4. On the other hand, ifcompletion of shooting is instructed, the procedure goes to step S24.

Steps S24 and S25 are provided to select an image by comparing previousand/or following frames when a scene cannot be determined to be a goodshot or not on a frame basis.

In other words, in step S24, the coordinates of the radial contour tipare compared with the coordinates of the face position to determine thescene in which the distance therebetween is longest to be a candidateimage. In this case, a scene of a subject stretching out his arms or legacross the frame is selected as a good shot candidate. This is suitablefor taking pictures of a fast moving subject. Then, in step S25, a goodshot candidate is selected based on the magnitude of a change of anobject adjacent to the radial contour (ball 57 in this case) in a scenechange, for example, where the ball 57 has disappeared from the frame asshown in FIGS. 5B and 5C. In this case, since the previous scene (forexample, FIG. 5B in FIGS. 5A to 5C) is selected, there is a highpossibility that the most impressive moment will be selected. Note thatif the procedure shifts to step S23 via step S3, since no face detectionis performed, the processing steps S24 and S25 are not executed inpractice.

Further, in step S26, the shot image is recorded.

Thus, good shot candidate images are detected during and after shooting.Then, when the user changes a mode switch (11 c or the like in FIG. 1)to the playback mode, the procedure shifts from step S27 to step S32.

In step S32, the presence or absence of a candidate image is determined.As a result, if there is a candidate image, the procedure goes to stepS33 to display the image preferentially. On the other hand, if there isno candidate image, the procedure shifts to step S34 to display ascandidate images four frames obtained by dividing sequential shots ateven intervals. Here, each shot image is determined as to whether it isa candidate image or not, and the determination result is recorded asattribute information for each image. Alternatively, only linkinformation indicating a link between each shot image and the presenceor absence of a candidate image can be recorded as a control fileseparately from the shot image.

Then, in step S35, a message to inform the user whether to erase imagesother than the displayed image appears, for example, on the display part27. If the user operates the camera to erase the images other than thedisplayed image, the procedure goes to step S36 to execute theoperation. As a result, needless images are deleted from the memory 23not to cause a problem for the next shooting. On the other hand, if theuser instructs the camera in step S35 not to erase the images other thanthe displayed image, step S36 is skipped. Thus, only the good shots areleft to always keep enough memory space in the memory 23, therebyavoiding such a problem that shooting is interrupted due to the shortageof free memory space in the memory 23.

Further, in step S37, it is determined whether the user operates thecamera to change images to be displayed. Here, if the user operates thecamera to change images to be displayed, the procedure goes to step S38to change images to be displayed. On the other hand, if the change ofimages to be displayed is not instructed in step S37, step S38 isskipped.

Thus the sequence is completed.

According to the first embodiment, there can be provided a cameracapable of capturing a rapid movement of a person properly.

In other words, when the camera performs rapid sequential shooting, onlyimages deemed appropriate as good shots are automatically selected fromamong a large number of sequential images so that the selected imagescan be recorded preferentially. Thus, there can be provided such acamera in a simple structure that can analyze the movement or shape of asubject to determine a pose deemed most worthy to be recorded, and hencethat enables even beginners to get their best shots.

Note that only the candidate images can be recorded in step S26. In thiscase, since the candidate images can be selected during shooting, ifimages other than the candidate images are not stored, more good shotscan be captured at high speed without undue waste.

Further, in this embodiment, a candidate image once obtained is furtherdetermined and confirmed using other criteria. However, the presentinvention is not limited thereto, and the candidate image once confirmeddoes not necessarily need to be further determined using other criteria.If the further determination is not made, processing speed can befurther increased though the number of candidate images is restricted.

Second Embodiment

A second embodiment of the present invention will next be described.

In the second embodiment to be described below, since the basicstructure of the camera is the same as that of the aforementioned firstembodiment, the same components are given the same reference numeralsand their description is omitted to avoid duplicated description. Thefollowing describes only the different points.

FIG. 10 is a flowchart for explaining the shooting operation of thecamera in the second embodiment of the present invention.

In this second embodiment, the skin tone of a subject is identifiedaccording to the results of face detection during shooting (or prior toshooting), and based on the skin tone, if a portion other than the facehas the same skin tone and is about half as large in size as the face,the portion with the skin tone is determined as a hand (or back of ahand). The skin tone is determined from color information of capturedimage data, for example, based on a distribution of R/G and B/G ratiosof the captured image data among R (red color information), G (greencolor information), and B (blue color information) in a predeterminedarea inside a screen. The skin tone is determined by a program executedinside the MPU. In this sense, the MPU can be referred to as a skin tonedetection part. Thus, if frequently shot poses of a subject as shown inFIGS. 11A to 11C are detected, these images are selected as good shotcandidates.

When this sequence is started, face detection is first performed in stepS41 to detect the position and size of a face portion in the screen.Then, in step S42, skin tone detection is performed from the color toneof the face portion detected in step S41. Further, in step S43, asubroutine “full-length figure determination” is executed. The detailsof this subroutine “full-length figure determination” will be describedlater.

Then, in step S44, it is determined whether the subject is a half-lengthor full-length figure from the position and size of the face of thesubject. If it is determined to be half-length shooting, the proceduregoes to step S45 to detect a skin tone portion in the vicinity of theface based on the determined skin tone of the face portion. Then, thesize of the detected skin tone portion is compared with the size of theface portion, and if it is about half as large in size as the faceportion, it is then determined in step S46 whether the size of the skintone portion corresponds to the size of a hand.

If the size of the skin tone portion corresponds to the size of a hand,the procedure shifts to hand position determination processing startingfrom step S47. In step S47, it is determined, for example, as shown inFIG. 11A, whether the position of a detected hand 60 b lies under a face60 a of a subject 60. Here, if it is determined that the position of thedetected hand lies under the face portion, the procedure goes to stepS48 to select the pose as a good shot candidate image. On the otherhand, if it is determined that the position of the detected hand doesnot lie under the face portion, step S48 is skipped.

Similarly, in step S49, it is determined, for example, as shown in FIG.11B, whether the position of the detected hand 60 b is beside the face60 a of the subject 60. Here, if it is determined that the position ofthe detected hand is beside the face portion, the procedure goes to stepS50 to select the pose as a good shot candidate image. On the otherhand, if it is determined that the position of the detected hand is notbeside the face portion, step S50 is skipped.

Then, in step S51, it is determined, for example, as shown in FIG. 11C,whether hands 60 b of the subject 60 are folded. Here, if it isdetermined that the hands of the subject are folded, the procedure goesto step S52 to select the pose as a good shot candidate image. Thedetermination of whether both hands of the subject are folded can bemade by detecting a hand portion 62 having the same color tone as a faceportion 61 as shown in FIG. 12 and detecting contours by sharpnessemphasis processing or the like in the same manner as in the firstembodiment to determine whether two contours are radially extended fromthe hand portion.

On the other hand, if it is determined in step S44 that it is nothalf-length (upper body) shooting, it is considered that full-lengthshooting is being done, and the procedure shifts to step S53. In stepS53, the skin tone portion determined in the skin tone determinationstep S42 is determined as to whether it lies downward on the screen. Inthis case, a portion having contours extended downward is considered tobe legs of the subject, and if a color other than the skin tone colordoes not appear between both legs, the procedure goes to step S54. Instep S54, for example, as shown in FIG. 1D, a pose of the subjectcrossing her legs is selected as a candidate image.

Note that the determination of whether both hands are folded can be madeby prestoring, in the flash ROM 15, templates indicating coordinatepositions or areas as reference conditions to compare each posetherewith in the same manner as in the aforementioned first embodiment.Of course, the pose can be determined further using data indicating theresults of tracking changes in the coordinates or area together with thetemplates.

Thus, only the candidate images that meet predetermined conditions canbe extracted from a large number of shot images, and this makes it easyto obtain good shot candidate images.

The subroutine “full-length figure determination” in step S43 of theflowchart of FIG. 10 is executed from the following point of view.

As shown in FIG. 13, suppose that the coordinates of a face 64 a of aperson 64 in a screen is determined as (x_(k), y_(K)) on the basis ofthe bottom left as a point of origin to determine the size of the faceas Δy. In this case, the distance from the bottom of the screen to thetop of the head is about y_(K)+Δy/2.

Under this condition, if the proportion of head to total body length isset to 1:6 to calculate 6×Δy, the coordinate position of the tiptoe isdetermined as y_(A). When the value y_(A) is larger than the point oforigin y=0, it can determined to be a full-length figure, while when thevalue y_(A) is smaller, it can be determined that the image does notinclude up to the subject's tiptoes. Of course, the proportion variesfrom person to person, or depending on whether the subject is a child oradult. Here, based on an average value, a calculation method as shown inthe following equation is employed:y _(A) =y _(K)+(Δy/2)−6Δy  (1)

FIG. 14 is a flowchart for explaining the subroutine “full-length figuredetermination” to determine whether the subject is a full-length orhalf-length figure from the above-mentioned point of view.

When the subroutine is started, the face coordinates (x_(K),y_(K)) arefirst detected in step S61, and the size of the face Δy is then detectedin step S62. This detection is performed by a method like patternmatching using the shadows of eyes, nose, and mouth as used for personalauthentication or security camera.

In step S63, the position of the foot (tiptoe) y_(A) is determinedaccording to the equation (1). Then, in step S64, it is determinedwhether the position of the foot (tiptoe) y_(A) is inside or outside ofthe screen. Here, if y_(A) is zero or larger, the procedure goes to stepS65 to process the image as a full-length figure. On the other hand, ify_(A) is less than zero, the procedure goes to step S66 to process theimage as a half-length figure.

Third Embodiment

A third embodiment of the present invention will next be described.

In the third embodiment to be described below, since the basic structureof the camera is the same as that of the aforementioned first and secondembodiments, the same components are given the same reference numeralsand their description is omitted to avoid duplicated description. Thefollowing describes only the different points.

The techniques as described in the first and second embodiments can beeffectively used when the user wants to leave in the memory only imagesdepicting appropriate and timely scenes selected from among a pluralityof images repeatedly shot at high speed.

However, as shown in FIG. 15, a time period Δt2 is required from wheneach image is captured and displayed until completion of good shotdetermination. If the start of the most timely shooting is decided fromthe determination criteria for the image just captured, the shootingtiming is affected by the time lag Δt2. Therefore, in the thirdembodiment, all captured images are once recorded, and as a result ofgood shot determination on each image, a priority flag is added to acorresponding image file.

In this case, there is no time lag between the time of recording eachimage in the above-mentioned manner and the time required to determinethe image. Further, good shot images can be displayed preferentiallyafter completion of shooting.

In order to effectively display an image determined to be a good shot,display control is so performed that the captured images are cascaded ondisplay part 27, with the image determined to be a good shot displayedon the top as shown in FIG. 17A. Alternatively, as shown in FIG. 17B,the image determined to be a good shot can be displayed at the center ofthe screen as the largest image, with the others displayed around it asthumbnail images. On this display screen, the user can determine thenecessity of images other than that determined as a good shot toinstruct deletion if the user determines them to be unnecessary, therebydeleting all but the good shot image.

FIG. 16 is a flowchart of the above-mentioned processing. Image datacaptured in step S71 undergoes image processing in step S72. Then, theimage data processed into a recording data format is recorded in thememory in step S73. After that, a live view image is displayed in stepS74.

Then, in step S75, good shot determination is performed from image datadisplayed in step S74. This determination is performed using thedetermination method described in the first or second embodiment. If itis determined in step S75 to be a good shot image, the procedure goes tostep S76 in which a flag indicating a good shot image is recorded in acorresponding image file as supplementary information. After that, instep S77, it is determined whether the turn-off of the release isinstructed. If it is determined that turn-off of the release isinstructed, the image capturing is completed, and in step S78, the flagadded image is displayed on the display part 27 in preference to theother captured images. After that, a priority image is selected in stepS79. On the other hand, if it is determined in step S77 that shooting isongoing, the procedure returns to step S71 to continue shooting.

In step S79, processing as shown in FIG. 18 is performed. In otherwords, it is determined in step S80 whether to delete all recordedimages but the priority image. This instruction is determined accordingto a user's operation based on the images and/or information displayedon the display part 27. If it is determined that all but the priorityimage are to be deleted, all shot and recorded image data but thepriority image are deleted (erased) from the memory in step S81.

As another application of priority selection on the screen, if the imagedetermined to be a good shot is not desired one, a good shot image canbe manually selected form other images as shown in FIG. 19.

In this case, the image determined to be a good shot is displayedfull-screen as shown in FIG. 17C so that the user can confirm that onlythis image is recorded so far. This allows the user to take a picturemore surely at the best timing without wasted consumption of memory.

In the flowchart of FIG. 19, it is first determined in step S82 whetherto delete all recorded images but the priority image. If it isdetermined that all but the priority image are to be deleted, all shotand recorded image data but the priority image are deleted (erased) fromthe memory in step S83.

If deletion of all recorded images but the priority image is notselected in step S82, the procedure goes to step S84 to allow the userto manually select images to be deleted. This selection is made in sucha manner that the user operates a key to select each image to be deletedfrom the displayed thumbnail images as shown in FIG. 17B and presses anaccept (OK) key to confirm the selection.

After the selection is confirmed, the display images are updated in stepS85 to display selected images to be stored, and in step S86, image dataselected as images to be deleted are deleted from the memory.

The images selected as images to be stored are stored with their framenumbers assigned in order of shooting. In the above example, althoughthe description is made about manual selection, the results of manualselection can also be stored. Then, when there is a correlation betweenthe good shot determination results and the manual selection results,for example, when there is a common tendency in the amount of shiftbetween shooting times, the amount of shift can be determined and fedback to the good shot determination so that the camera can learn fromthe feedback using a known method. In this case, an image captured attiming that more reflects user's preferences can be selected.

Further, in the embodiment, the image processing, image recording, imagedisplay, and good shot determination are performed in a time sequence,but the present invention is not limited thereto. For example, the goodshot determination and other processing can be performed in parallelwith each other using a system having two or more MPUs. Use of such asystem can perform these processing at a higher speed.

While there has been shown and described what are considered to bepreferred embodiments of the invention, it will, of course, beunderstood that various modifications and changes in form or detailcould readily be made without departing from the spirit of theinvention. It is therefore intended that the invention not be limited tothe exact forms described and illustrated, but constructed to cover allmodifications that may fall within the scope of the appended claims.

What is claimed is:
 1. A camera comprising: a face portion detectionpart configured to detect a face portion included in an image signalfrom the image signal output from an image pickup device; a referencevalue storing part configured to pre-store reference values forcoordinates of the face portion and reference values for coordinates ofa hand or leg, or reference values for an amount of change in thecoordinates of the hand or leg; a contour detection part configured todetect a contour of a subject from the image signal output from theimage pickup device; a determination part configured to selectcoordinates corresponding to a hand or leg from the contour detected bythe contour detection part, and determine (1) whether coordinates of thedetected face portion and the selected coordinates corresponding to thehand or leg match the reference values stored in the reference valuestoring part, or (2) whether the amount of change in the selectedcoordinates of the hand or leg exceeds reference value stored in thereference value storing part; and a control part configured to attach,when the determination part determines that the reference values arematched or exceeded, data on a processed image for identifying theprocessed image as a candidate image to be stored or displayedpreferentially.
 2. The camera according to claim 1 further comprising arecording medium for recording shot images, wherein the camera selectsthe candidate image from the shot images and stores the selectedcandidate image in the recording medium.
 3. The camera according toclaim 1, wherein the reference value storing part stores values for thecoordinates of the hand or leg as reference if the selected coordinatescorresponding to the hand or leg is (1) above the coordinates of thedetected face portion, or (2) below the coordinates of the detected faceportion.
 4. A camera having a sequential shooting function forsequentially shooting a plurality of images comprising: an image pickupdevice configured to capture an image of a subject; a face portiondetection part configured to detect a face portion of the subjectincluded in the image captured by the image pickup device; a referencevalue storing part configured to pre-store reference values forcoordinates of the detected face portion and reference values forcoordinates of a hand or leg, or reference values for an amount ofchange in the coordinates of the hand or leg; a contour detection partconfigured to detect a contour of a subject from the image signal outputfrom the image pickup device; a determination part configured to selectcoordinates corresponding to a hand or leg from the contour detected bythe contour detection part, and determine (1) whether coordinates of thedetected face portion and the selected coordinates corresponding to thehand or leg match the reference values stored in the reference valuestoring part, or (2) whether the amount of change in the selectedcoordinates of the hand or leg exceeds reference value stored in thereference value storing part; and a control part configured to attach,when the determination part determines that the reference values arematched or exceeded, data on a processed image of the plurality ofimages for identifying the processed image as a candidate image to bestored or displayed preferentially.
 5. The camera according to claim 4further comprising a recording medium for recording shot images, whereinthe camera selects the candidate image from the shot images and storesthe selected candidate image in the recording medium.
 6. The cameraaccording to claim 4, wherein the reference value storing part storesvalues for the coordinates of the hand or leg as reference if theselected coordinates corresponding to the hand or leg is (1) above thecoordinates of the detected face portion in at least one of theplurality of images, or (2) below the coordinate of the detected faceportion in at least one of the plurality of images.
 7. The cameraaccording to claim 4, wherein the reference value storing part storesvalues for the coordinates of the hand or leg as reference when theselected coordinates corresponding to the hand or leg reaches a positionfarthest from the coordinates of the detected face portion among theplurality of images.